Method and apparatus for video coding, predecoding, and video decoding for video streaming service, and image filtering method

ABSTRACT

A method and apparatus for video encoding, predecoding, and video decoding for video streaming services. The video encoding method includes encoding first and second video sequences into first and second bitstreams using scalable video coding, wherein at least one of resolution, frame rate, and image quality of the second video sequence is different from that of the first video sequence, and combining the first and second bitstreams into a super bitstream.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority from Korean Patent Application No.10-2004-0022985 filed on Apr. 2, 2004 in the Korean IntellectualProperty Office, and U.S. Provisional Patent Application No. 60/549,544filed on Mar. 4, 2004 in the United States Patent and Trademark Office,the disclosures of which are incorporated herein by reference in theirentirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and apparatus for videoencoding, predecoding, and reconstructing the original video sequencefor video streaming services, a bitstream structure, and an imagefiltering method.

2. Description of the Related Art

With the development of information communication technology includingthe Internet, a variety of communication services have been newlyproposed. One among such communication services is a Video On Demand(VOD) service. Video on demand refers to a service in which a videocontent such as movies or news is provided to an end user over atelephone line, cable or Internet upon the user's request. Users areallowed to view a movie without having to leave their residence. Also,users are allowed to access various types of knowledge via moving imagelectures without having to go to school or private educationalinstitutes.

Various requirements must be satisfied to implement such a VOD service,including wideband communications and motion picture compression totransmit and receive a large amount of data. Specifically, moving imagecompression enables VOD by effectively reducing bandwidths required fordata transmission. For example, a 24-bit true color image having aresolution of 640×480 needs a capacity of 640×480×24 bits, i.e., data ofabout 7.37 Mbits, per frame. When this image is transmitted at a speedof 30 frames per second, a bandwidth of 221 Mbits/sec is required toprovide a VOD service. When a 90-minute movie based on such an image isstored, a storage space of about 1200 Gbits is required. Accordingly,since uncompressed moving images require a tremendous bandwidth and alarge capacity of storage media for transmission, a compression codingmethod is a requisite for providing the VOD service under currentnetwork environments.

A basic principle of data compression is removing data redundancy.Motion picture compression can be effectively performed when the samecolor or object is repeated in an image, or when there is little changebetween adjacent frames in a moving image.

Known video coding algorithms for motion picture compression includeMoving Picture Experts Group (MPEG)-1, MPEG-2, H.263, and H.264 (orAVC). In such video coding methods, temporal redundancy is removed bymotion compensation based on motion estimation and compensation, andspatial redundancy is removed by Discrete Cosine Transformation (DCT).These methods have high compression rates, but they do not havesatisfactory scalability since they use a recursive approach in a mainalgorithm. In recent years, research into data coding methods havingscalability, such as wavelet video coding and Motion CompensatedTemporal Filtering (MCTF), has been actively carried out. Scalabilityindicates the ability to partially decode a single compressed bitstreamat different quality levels, resolutions, or frame rates.

FIG. 1 illustrates the configuration of a video streaming serviceprovider 100 using a video coding scheme supporting low scalability. Forconvenience of explanation, a video streaming service for a single videosequence will be described.

Referring to FIG. 1, the video streaming service provider 100 receives avideo sequence and performs video coding on the video sequence using acoding algorithm such as MPEG-1, MPEG-2, H.263, or H.264. A bitstreamobtained by coding the video sequence with these coding algorithms isnot scalable or supports little scalability. Thus, to provide videostreaming services at various spatial resolutions and frame rates, abitstream needs to be generated for each resolution and frame rate. Toaccomplish this, the video streaming service provider 100 includes aplurality of converters 110-2 through 110-n, each converting a videosequence into another video sequence with a lower spatial resolution and(or) a lower frame rate, a plurality of encoders 120-1 through 120-nencoding the video sequence or the video sequences subjected toconversion with a video coding algorithm into bitstreams, and a selector130 selecting one of the bitstreams with different spatial resolutionsand frame rates for transmission to a video decoder 140.

More specifically, a second converter 110-2 converts the received videosequence into a video sequence with a lower spatial resolution or (and)a lower frame rate by performing downsampling or frame rate reduction.MPEG-based downsampling results in smooth images. The resulting videosequence is then sent to a second video encoder 120-2. Similarly, athird converter 110-3 converts the video sequence and sends theresulting sequence to a third video encoder 120-3, and an n-th converter110-n transmits the video sequence to an n-th video encoder 120-n afterconversion.

A first video encoder 120-1 performs video coding on the video sequenceat the highest spatial resolution and highest frame rate. For example,the first video encoder 120-1 may receive the video sequence with704×576 resolution and 60 Hz frame rate and encode the video sequenceinto a bitstream with 704×576 resolution and 60 Hz frame rate. Thebitstream obtained by coding while maintaining the same resolution andframe rate as the original video sequence can be provided to a user whena sufficient network bandwidth is available to support it. For example,if 6 Mbps network bandwidth is stably available, the bitstream generatedby the first video encoder 120-1 can be provided to the user. Thebitstream provided to the user is decoded by the video decoder 140 toreconstruct the original video sequence with 704×576 resolution and 60Hz frame rate.

The second video encoder 120-2 encodes a video sequence with a lowerspatial resolution and (or) a lower frame rate than that encoded by thefirst video encoder 120-1 into a bitstream. Similarly, the third videoencoder 120-3 performs video encoding at different spatial resolutionand (or) frame rate than the first and second video encoders 120-1 and120-2 and generates a bitstream. In this way, the first through the n-thvideo encoders 120-1 through 120-n generate bitstreams with differentspatial resolutions and (or) frame rates from the same video sequence.

The selector 130 provides a bitstream having a spatial resolution and aframe rate requested by the user (video decoder 140) to the videodecoder 140. When a sufficient network bandwidth is available, the usercan make a request for a video with a high spatial resolution and a highframe rate, and the video streaming service provider 100 delivers abitstream with the high spatial resolution and the high frame rateselected by the user to the user. If the network bandwidth is notstable, a video sequence reconstructed by the video decoder 130 from abitstream coded at high resolution and high frame rate can be easilydisrupted during playback. In this case, the user can request abitstream coded at lower resolution and (or) lower frame rate from thevideo streaming service provider 100.

The video decoder 140 receives a bitstream corresponding to each videosequence from the video streaming service provider 100 for decoding. Forexample, in order to reconstruct a video sequence, an MPEG-2 codedbitstream can be decoded using an MPEG-2 decoding algorithm while anH.264 coded bitstream can be decoded using H.264 decoding scheme.

A video streaming service provider using non-scalable or low scalabilityvideo coding algorithm like in FIG. 1 must perform a plurality of videocoding processes on the same video sequence with various spatialresolutions and frame rates according to network environment or user'srequest. As a result, a plurality of bitstreams are generated for thesame video sequence. Generating a bitstream at each resolution and framerate requires a great deal of computational capacity. Furthermore,services delivering video streams to users at various spatialresolutions and frame rates, which are commonly known as simulcastingservices, require high capacity storage media for storing generatedbitstreams.

FIG. 2 schematically illustrates the configuration of a video streamingservice provider 200 using a wavelet-based scalable video coding scheme.For convenience of explanation, video coding for a single video sequencewill be described.

Referring to FIG. 2, the video streaming service provider 200 includes ascalable video encoder 210 encoding a video sequence and a predecoder220. The scalable video encoder 210 uses a video coding algorithm havingscalability to generate a scalable bitstream. In currently knownscalable video coding algorithms, spatial scalability can be attained bywavelet transform, temporal scalability can be attained by MotionCompensated Temporal Filtering (MCTF), unconstrained MCTF (UMCTF) orSuccessive Temporal Approximation and Referencing (STAR), and Signal toNoise Ratio (SNR) scalability can be attained by embedded quantization.

The bitstream obtained by encoding the video sequence through thescalable video encoder 210 is predecoded by the predecoder 220.Predecoding is a process of truncating some bits of a scalablebitstream. The bitstream may be predecoded into a bitstream with a lowerspatial resolution, a lower frame rate, or a lower image quality than anoriginal bitstream. When the video decoder 230 at the user side requestsa video sequence with specific resolution and frame rate from the videostreaming service provider 200, the predecoder 220 in the videostreaming service provider 200 truncates some bits of the bitstream andtransmits the resulting bitstream to the video decoder 230. The videodecoder 230 decodes the bitstream and reconstructs a video sequence withthe requested resolution and frame rate.

Using a scalable video coding algorithm for a video streaming service inthis way allows simulcasting of a single bitstream obtained from asingle video sequence at various resolutions and frame rates. However,currently known scalable video coding algorithms do not offer highquality bitstreams at all resolutions. For example, the highestresolution video can be reconstructed with high quality, but a lowresolution video cannot be reconstructed with satisfactory quality. Morebits can be allocated for video coding of the low resolution video toimprove its quality. However, this will degrade the coding efficiency.

As described above, the video streaming service shown in FIG. 1 canprovide a bitstream optimized at every resolution, but may wastecomputational capacity and storage space. On the other hand, the videostreaming service shown in FIG. 2 is able to provide bitstreams havingvarious resolutions and frame rates using a single bitstream, but mayoffer poor image quality at some resolutions or degrade codingefficiency to improve image quality. Therefore, there is an urgent needfor a video coding scheme for video streaming service deliveringsatisfactory image quality and high video coding efficiency by achievinga good trade-off between the coding efficiency and reconstructed imagequality.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus for videoencoding, predecoding, and video decoding for video streaming services.

The present invention also provides a method for increasing theefficiency of a video coding algorithm for video streaming services.

The present invention also provides a method for improving the imagequality of a video sequence provided by a video streaming service.

The above stated object as well as other objects, features andadvantages, of the present invention will become clear to those skilledin the art upon review of the following description, the attacheddrawings and appended claims.

According to an aspect of the present invention, there is provided avideo encoding method including encoding first and second videosequences into first and second bitstreams using scalable video coding,wherein at least one of resolution, frame rate, and image quality of thesecond video sequence is different from that of the first videosequence; and combining the first and second bitstreams into a superbitstream.

According to another aspect of the present invention, there is provideda video encoding method including encoding first through n-th videosequences into first through n-th bitstreams wherein n is a naturalnumber greater than 1 and at least one of resolution, frame rate, andimage quality of each of the second through the n-th video sequences isdifferent from that of the first video sequence, and combining the firstthrough the n-th bitstreams together into a super bitstream.

According to still another aspect of the present invention, there isprovided a super bitstream format comprising a first bitstream generatedby encoding a first video sequence having the highest resolution, andsecond through n-th bitstreams generated by respectively encoding secondthrough n-th video sequences having resolution, frame rate, and imagequality, at least one of which is different from that of the first videosequence where n is a natural number greater than 1.

According to yet another aspect of the present invention, there isprovided a video encoder including a video encoding unit encoding firstthrough n-th video sequences wherein at least one of resolution, framerate, and image quality of each of the second through the n-th videosequences is different from that of the first video sequence, and asuper bitstream generator generating a super bitstream including thefirst through n-th bitstreams generated by the video encoding unit andnecessary header information.

According to a further aspect of the present invention, there isprovided a predecoding method including receiving a request for a videosequence with predetermined resolution, frame rate, and image quality,truncating a portion of a super bitstream including a plurality ofbitstreams with different resolutions and frame rates associated withthe requested video sequence, so that the super bitstream has the sameresolution, frame rate, and image quality as requested, and transmittingthe resulting super bitstream to a decoder.

According to still another aspect of the present invention, there isprovided a predecoder including a request receiver receiving a requestfor a video sequence with predetermined resolution, frame rate, andimage quality, a super bitstream truncating portion truncating a portionof a super bitstream including a plurality of bitstreams with differentresolutions and frame rates associated with the requested videosequence, so that the super bitstream has the same resolution, framerate, and image quality as requested, and a transmitter sending theresulting super bitstream to a decoder.

According to a further aspect of the present invention, there isprovided a method for reconstructing a video sequence, includingreceiving a compressed video sequence, performing inverse quantizationand inverse transform on the compressed video sequence, andreconstructing an intraframe, filtering the intraframe so that theintraframe is used as a reference frame in reconstructing interframes,and reconstructing the interframes by using a filtered version of theintraframe as a reference.

According to yet another aspect of the present invention, there isprovided an apparatus for reconstructing the original video sequencefrom a compressed video sequence, the apparatus including a receiverreceiving the compressed video sequence; a decoder reconstructing anintraframe by decoding the compressed video sequence, and a filterfiltering the reconstructed intraframe so that the intraframe is used asa reference in reconstructing interframes, wherein the decoderreconstructs interframes by decoding the compressed video sequence usinga filtered version of the intraframe as a reference.

According to still another aspect of the present invention, there isprovided a method for filtering an image obtained by wavelet-baseddownsampling including performing wavelet-based upsampling on the imageobtained by the wavelet-based downsampling, and downsampling anupsampled version of the image using a predetermined scheme.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present inventionwill become more apparent by describing in detail exemplary embodimentsthereof with reference to the attached drawings in which:

FIG. 1 illustrates the concept of a conventional video streamingservice;

FIG. 2 illustrates the concept of another conventional video streamingservice;

FIG. 3 illustrates the concept of a video streaming service according toan exemplary embodiment of the present invention;

FIGS. 4A and 4B schematically shows the configurations of a scalablevideo encoder and an Advanced Video Coding (AVC) video encoder;

FIGS. 5A-5D show the comparison between video streaming services usingnon-scalable and scalable video coding schemes and other coding schemesaccording to embodiments of the present invention;

FIG. 6 is a diagram for explaining sharing of an intraframe according toa first embodiment of the present invention;

FIG. 7 is a diagram for explaining sharing of an intraframe according toa second embodiment of the present invention;

FIG. 8 is a diagram for explaining a method for generating a smoothreference frame according to an embodiment of the present invention;

FIG. 9 illustrates a process for providing a video streaming serviceaccording to an embodiment of the present invention; and

FIG. 10 shows the structure of a super bitstream according to anembodiment of the present invention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS OF THE INVENTION

The present invention will now be described more fully with reference tothe accompanying drawings, in which exemplary embodiments of theinvention are shown.

In the embodiments of the present invention, scalable video coding isused to generate bitstreams with various spatial resolutions and framerates. That is, the scalable coding allows a single scalable bitstreamto be partitioned into bitstreams having various spatial resolutions andframe rates.

Currently known scalable coding algorithms cannot ensure reconstructionof good quality video sequence at every resolution level. Thus, whilethe present invention basically uses a scalable video coding algorithmto perform video coding on a video sequence, it does not obtain videosequences with all resolutions and frame rates from a single scalablecoded bitstream. In one embodiment, two or more scalable bitstreams aregenerated for a single video sequence. In another embodiment, a scalablebitstream and an MPEG-based bitstream are generated for a single videosequence. The generated bitstreams are combined together into a superbitstream.

FIG. 3 illustrates the concept of a video streaming service according toan embodiment of the present invention. It is assumed that a videostreaming service is provided for a single video sequence (content).

Referring to FIG. 3, a video streaming service provider 300 generates aplurality of scalable bitstreams and combines them into a single superbitstream for transmission to a video decoder 350.

The video streaming service provider 300 includes a converter 310, avideo encoder 320, a super bitstream generator 330, and a predecoder340.

The video encoder 320 encodes a plurality of video sequences withdifferent resolutions, frame rates, and image qualities into bitstreams.In one embodiment, the video encoder 320 uses only a scalable videocoding algorithm to generate bitstreams from the video sequences. Inanother embodiment, the video encoder uses both scalable video codingscheme and a Discrete Cosine Transform (DCT)-based non-scalable videocoding scheme to generate bitstreams.

As shown in FIG. 3, the video encoder 320 using only the scalable videocoding scheme consists of first through n-th scalable video encodingunits 320-1 through 320-n that receive a number n of video sequenceswith different spatial resolutions and frame rates obtained byconverting a single video sequence (content) and generate n bitstreams.The first through the n-th scalable video encoding units 320-1 through320-n may be separate devices or be integrated into a single device.

The first scalable video encoding unit 320-1 performs scalable videoencoding on a video sequence at the highest spatial resolution andhighest frame rate, and the second scalable video encoding unit 320-2performs scalable video encoding at lower resolution and lower framerate than the first scalable video encoding unit 320-1. In the samemanner, the remaining scalable video encoding units 320-3 through 320-nperform scalable video coding at different spatial resolutions and framerates. Since reconstructing a video sequence with lower resolution thanthe generated scalable bitstream will degrade image quality moreseverely than with lower frame rate, it is desirable for each of thescalable video encoding units 320-1 through 320-n to generate a scalablebitstream with a different resolution.

A converter 310 that receives a video sequence converts the videosequence into video sequences having lower resolutions or lower framerates. More specifically, a second converting unit 310-2 converts thevideo sequence into another video sequence with the same spatialresolution and frame rate as a scalable bitstream to be generated by thesecond scalable video encoding unit 320-2. Similarly, the remainingconverting units 310-3 through 310-n respectively convert the videosequence into video sequences with different resolutions and frame ratesfor encoding in the third through the n-th scalable video encoding units320-3 through 320-n. The video sequence can be converted into a lowerresolution by either wavelet-based downsampling or MPEG-baseddownsampling. Alternatively, a frame may be downsampled using awavelet-based method, a downsampled version of the frame may beupsampled using the wavelet-based method, and an upsampled version maybe downsampled using an MPEG-based scheme. Wavelet-based downsamplingmeans selecting a low-pass subband among low- and high-pass subbandsobtained by spatially compressing an image with a wavelet-based method.This wavelet-based downsampling (upsampling) can also be performed withencoding operations by each of the scalable video encoding units 320-1through 320-n.

The n video sequences are converted into n scalable bitstreams withdifferent spatial resolutions and frame rates by the scalable videoencoding units 320-1 through 320-n. The video streaming service provider300 further includes the super bitstream generator 330 combining thegenerated n scalable bitstreams into a single super bitstream.

The predecoder 340 sends a bitstream with requested resolution and framerate to the video decoder 350. In one embodiment, the predecoder 340selects a bitstream with the requested resolution among the nbitstreams, truncates unnecessary bits so that the selected bitstreamhas the same frame rate as requested, and transmits the resultingbitstream to the video decoder 350. The predecoder 340 may receive avideo sequence and a request for resolution and frame rate associatedwith the video sequence to be reconstructed directly from the videodecoder 350 or from the video streaming service provider 300 thatreceives the video sequence and the request from the video decoder 350.To accomplish this, the predecoder 340 includes a request receiver (notshown) receiving the request and a bitstream truncating portion (notshown) cutting bits of a bitstream.

In another embodiment, the predecoder 340 selects a bitstream thatmatches the video sequence and the resolution and frame rate associatedwith the video sequence received by the request receiver among the superbitstream containing the n bitstreams, truncates the bitstreams otherthan the selected bitstream and unnecessary bits of the selectedbitstream, and sends the resulting super bitstream to the video decoder350. If a scalable bitstream with the requested resolution does notexist within the super bitstream, one of the higher resolution scalablebitstreams is selected and unnecessary bits of the selected bitstreamare truncated for transmission to the video decoder 350. In this case,the selected scalable bitstream may have resolution closest to therequested resolution. Furthermore, if a scalable bitstream with therequested resolution and frame rate exists within the super bitstream,the predecoder 340 may convert the selected bitstream into a lower SNRbitstream by truncating the same before transmission to the videodecoder 350.

In one embodiment, the predecoder 340 may be implemented separately fromthe video encoder 320. In this case, the predecoder 340 acts as thevideo streaming service provider 300. That is, the predecoder 340receives a request for a video sequence with specific resolution andframe rate from the video decoder 350 and selects one among the superbitstream containing the plurality of previously encoded bitstreams,truncates some bits of the selected bitstream and the remainingbitstreams, and transmits the resulting super bitstream to the videodecoder for reconstruction of the video sequence with the requestedresolution and frame rate.

In another embodiment, the predecoder 340 may be located together withthe video decoder 350 at the user side. In this case, the video decoder350 that receives a super bitstream from the video streaming serviceprovider 300 sends the super bitstream to the predecoder 340 that thenselects a bitstream from the super bitstream and truncates some bits ofthe selected bitstream and the remaining unnecessary bitstreams in sucha manner as to reconstruct a video sequence with the resolution andframe rate desired by the video decoder 350.

The above-described components are functional modules and perform thetasks described above. The term ‘module’, as used herein, means, but isnot limited to, a software or hardware component, such as a FieldProgrammable Gate Array (FPGA) or Application Specific IntegratedCircuit (ASIC), which performs certain tasks. A module mayadvantageously be configured to reside on the addressable storage mediumand configured to execute on one or more processors. Thus, a module mayinclude, by way of example, components, such as software components,object-oriented software components, class components and taskcomponents, processes, functions, attributes, procedures, subroutines,segments of program code, drivers, firmware, microcode, circuitry, data,databases, data structures, tables, arrays, and variables. Thefunctionality provided for in the components and modules may be combinedinto fewer components and modules or further separated into additionalcomponents and modules. In addition, the components and modules may beimplemented such that they execute one or more computers in acommunication system.

FIGS. 4A and 4B schematically show the configurations of a scalablevideo encoder 410 and an Advanced Video Coding (AVC) video encoder 460,respectively.

Referring to FIG. 4A, the video scalable encoder 410 includes amotion-compensated temporal filter 420, a wavelet transformer 430, andan embedded quantizer 440. The scalable video encoder 410 receives aninput video sequence that is split into several groups of pictures(GOPs), each being the smallest encodable unit. A GOP consists of aplurality of frames, e.g., 2, 4, 8, 16, or 32 frames. Various algorithmsknown as the scalable video coding schemes provide higher videocompression efficiency as the number of frames in a GOP (‘GOP size’)increases. However, increasing the GOP size also increases algorithmdelay from video encoding to decoding. On the other hand, decreasing theGOP size decreases the algorithm delay at the expense of sacrificingcompression efficiency.

The motion-compensated temporal filter 420 removes temporal redundanciesbetween frames in each GOP using commonly known algorithms such asMotion Compensated Temporal Filtering (MCTF), Unconstrained MCTF(UMCTF), or Successive Temporal Approximation and Referencing (STAR).These algorithms not only remove temporal redundancies between framesbut also achieve video coding that provides temporal scalability.

The wavelet transformer 430 converts the frames from which the temporalredundancies have been removed using a wavelet transform algorithm andremoves spatial redundancies therefrom. The wavelet transform algorithmis also used in JPEG2000 standard, and decomposes a frame into low- andhigh-pass subbands. The low-pass subband image is similar to a reducedversion of the original image.

The embedded quantizer 440 performs embedded quantization on transformcoefficients obtained by applying wavelet transform on the frames.Entropy encoding is performed to convert the transform coefficientssubjected to embedded quantization into a bitstream.

The present invention also uses another video coding algorithm togenerate some or all of bitstreams for a video sequence. An AVCalgorithm also known as H.264 or MPEG-4 Part 10 uses DCT fortransformation and delivers the highest compression efficiency currentlyavailable. In the embodiment of the present invention using non-scalablevideo coding scheme, the AVC is desirable.

The AVC video encoder 460 includes a motion-compensated predictor 470, aDCT transformer 480, and a quantizer 490. The motion-compensatedpredictor 470 removes temporal redundancies present within frames makingup a video sequence. The AVC supports various block sizes such as 4×4,4×8, 8×4, 8×8, 16×8, 8×16, and 16×16 sub-blocks for removal of thetemporal redundancies, thereby achieving high compression efficiency.

The DCT transformer 480 performs a DCT transform to divide each of theframes from which the temporal redundancies have been removed by themotion-compensated predictor 470 into a number of macroblocks. Incontrast to the scalable video coding scheme using wavelet transform,the AVC scheme cannot support spatial scalability since it applies DCTtransform to each macroblock. The quantizer 490 quantizes transformcoefficients subjected to DCT, which is then entropy encoded into abitstream. The video encoder 320 shown in FIG. 3 may include thescalable video encoder 410, the AVC video encoder 460, or other videoencoders.

Meanwhile, a decoder (not shown) performs an inverse operation of theencoder 410 or 460 to reconstruct a video sequence. That is, the decoderthat receives the bitstream (compressed video sequence) sequentiallyperforms inverse quantization (or inverse embedded quantization),inverse spatial transform (wavelet transform or DCT transform), andinverse motion compensated temporal filtering or inverse motioncompensated prediction on the bitstream to reconstruct a video sequence.

FIGS. 5A-5D respectively show video streaming services using anon-scalable coding scheme, scalable video coding scheme, and othercoding schemes according to first and second embodiments of the presentinvention.

For a video streaming service using the non-scalable video codingscheme, a bitstream is required for each resolution and frame rate toprovide a video sequence (content) to a user at various resolutions andframes rates. For example, to provide three kinds of video streamingservices with 704×576 resolution and 60 Hz frame rate, 352×288resolution and 30 Hz frame rate, and 176×144 resolution and 15 Hz framerate, video coding is performed on a video sequence at the 704×576resolution and 60 Hz frame rate, the 352×288 resolution and 30 Hz framerate, and the 176×144 resolution and 15 Hz frame rate to generate firstthrough third bitstreams 511 through 513, respectively. For example,when the user makes requests for the 704×576 resolution and 60 Hz framerate, the 352×288 resolution and 30 Hz frame rate, and 176×144resolution and 15 Hz frame rate, a video streaming service provideroffers the first through third bitstreams 511 through 513 respectivelyhaving the 704×576 resolution and 60 Hz frame rate, the 352×288resolution and 30 Hz frame rate, and the 176×144 resolution and 15 Hzframe rate.

As shown in FIG. 5A, offering the first through third bitstreams 511through 513, respectively, requires stable 6 Mbps, 750 Kbps, and 128Kbps network bandwidths. Thus, the user should select a bitstream withresolution and frame rate to match the network bandwidth available. Thatis, when a stable network bandwidth of greater than 6 Mbps is ensured,the user is able to receive the video streaming services with the704×576 resolution and 60 Hz frame rate, the 352×288 resolution and 30Hz frame rate, and the 176×144 resolution and 15 Hz frame rate. On theother hand, when the available network bandwidth is greater than 750Kbps but less than 6 Mbps, the user is able to receive only the videostreaming services with the 352×288 resolution and 30 Hz frame rate andthe 176×144 resolution and 15 Hz frame rate. Similarly, when a bandwidthof 128 Kbps is ensured, the user is able to receive only the videostreaming service with the 176×144 resolution and 15 Hz frame rate.

When a stable bandwidth of 5 Mbps is available, the user cannot receivethe video streaming service with the 704×576 resolution and 60 Hz framerate. Thus, to satisfy user's need for higher resolution and frame rate,the video streaming service provider needs to encode a video sequence atresolution and frame rate optimized for the 5 Mbps bandwidth. That is,the streaming service using the non-scalable video coding algorithmposes a limitation in that video coding must be performed for eachresolution and frame rate in order to provide video streaming serviceswith various resolutions and frame rates according to network conditionor user's preference. An alternative solution to this limitation using ascalable video coding algorithm will now be described.

Referring to FIG. 5B, for a scalable video streaming service, a singlescalable bitstream 520 is generated for a single content (videosequence). The scalable bitstream 520 may be easily predecoded by apredecoder into a bitstream with either or both of lower resolution andlower frame rate.

For example, the scalable bitstream 520 with 704×576 resolution and 60Hz frame rate requiring a 6 Mbps bandwidth may be predecoded into abitstream with a frame rate of 30 Hz or 15 Hz without a change in theresolution.

Furthermore, the scalable bitstream 520 with 704×576 resolution and 60Hz frame rate requiring a stable 6 Mbps bandwidth may be predecoded intoa bitstream with lower resolution and lower frame rate, e.g., abitstream with 352×288 resolution and 30 Hz frame rate requiring astable 750 Kbps bandwidth, a bitstream with 352×288 resolution and 15 Hzframe rate requiring a stable 192 Kbps bandwidth, or a bitstream with176×144 resolution and 15 Hz frame rate requiring a stable 128 Kbpsbandwidth.

In addition, the scalable bitstream 520 may be predecoded into a lowerimage quality bitstream without changes in resolution and frame rate.For example, when the scalable bitstream 520 is converted into abitstream with the 352×288 resolution and 30 Hz frame rate requiring astable bandwidth of 384 Kbps before transmission to the user, a videosequence reconstructed from the bitstream has lower image quality thanfrom a bitstream with the same resolution and frame rate but requiring a750 Kbps bandwidth. Similarly, the scalable bitstream may be convertedinto a bitstream with the 176×144 resolution and 15 Hz frame raterequiring a stable 128 Kbps bandwidth or a lower quality bitstream withthe same resolution and frame rate requiring a stable 64 Kbps bandwidthbefore transmission to the user.

In contrast to the video streaming service using the non-scalable videocoding algorithm shown in FIG. 5A, the video streaming service using thescalable video coding algorithm enables transmission of bitstreams withvarious resolutions, frame rates, and image qualities using the singlescalable bitstream 520. That is, using the scalable video codingalgorithm makes it easy to provide various video streaming servicesaccording to a network condition or performance of a user's device.However, the currently known scalable video coding algorithms cannotobtain good quality reconstructed video sequences at all resolutions.For example, the scalable video coding algorithm may obtain good qualityreconstructed video sequence at the 704×576 resolution, but not at the176×144 resolution.

To solve this problem, the video streaming service according to thefirst embodiment of the present invention uses a coding scheme thatinvolves generating a plurality of scalable bitstreams and combiningthem together into a single super bitstream instead of predecoding ascalable bitstream in order to provide a video sequence at variousresolutions, frame rates, and image qualities.

Referring to FIG. 5C, first through third scalable bitstreams 531through 533 are generated at different resolutions from a single videosequence and then combined into a super bitstream 530. Alternatively,one or a plurality of scalable bitstreams with different resolutions andone or a plurality of non-scalable bitstreams may be generated for thesingle video sequence, which will be described later with reference toFIG. 5D.

The first scalable bitstream 531 is generated from a video sequence atthe highest resolution and frame rate (e.g., 704×576 resolution and 60Hz frame rate). The second scalable bitstream 532 is generated from thesame video sequence at lower resolution and lower frame rate than thefirst scalable bitstream 531 (e.g., 352×288 resolution and 30 Hz framerate). The third bitstream has the lowest resolution and frame rate(176×144 resolution and 15 Hz frame rate). In one embodiment, when auser requests transmission of a frame with 352×288 resolution over aavailable bandwidth of 384 Kbps, the second scalable bitstream 530 isconverted into a lower quality bitstream with the same resolution andframe rate by truncating unnecessary bits of the second scalablebitstream 530 with a predecoder before transmission to the user.

Of course, the bitstream with the 352×288 resolution and the 30 Hz framerate to be transmitted over the 384 Kbps bandwidth can be obtained fromthe first scalable bitstream 531. However, the resulting bitstream mayhave lower quality than that obtained from the second scalable bitstream532. While it is described that the resolutions and frame rates of thesecond and third scalable bitstreams 532 and 533 are half, quarter, andeighth that of the first scalable bitstream 531, they may be a third orsixth of the same.

In another embodiment, the first and second scalable bitstreams 531 and532 may respectively have 704×576 and 600×480 resolutions. In this case,the bitstream with a 352×288 resolution may be obtained by predecodingthe first scalable bitstream 531 while a bitstream with a 300×240resolution may be obtained by predecoding the second scalable bitstream532.

The video streaming service shown in FIG. 5C employs scalable videocoding to achieve scalability and uses a plurality of scalablebitstreams to reduce degradation in image quality compared to the videostreaming service shown in FIG. 5B in which a single bitstream isconverted into bitstreams with various resolutions and frame rates.Thus, using the coding scheme according to the first embodiment of thepresent invention allows a video streaming service for a video sequenceat various resolutions, frames rates, and image qualities, which iscalled simulcasting, while ensuring the appropriate and acceptable imagequality for each resolution level.

Referring to FIG. 5D, the coding scheme according to the secondembodiment is used to generate a plurality of bitstreams containing anon-scalable bitstream. That is, a super bitstream 540 consists of firstand second scalable bitstreams 541 and 542 and a third non-scalablebitstream 543. While the first and second scalable bitstreams 541 and542 are generated using a scalable video coding scheme, the thirdbitstream 543 is generated using a non-scalable video coding scheme suchas AVC.

Meanwhile, the video streaming services according to the embodiments ofthe present invention require more data storage space than using thesingle scalable video stream shown in FIG. 5B. A method for reducingthis overhead will now be described with reference to FIGS. 6 and 7,assuming that a super bitstream contains two scalable bitstream.

FIG. 6 is a diagram for explaining sharing of an intraframe according toa first embodiment of the present invention. Frames in the videosequence used in video coding can be divided into intraframes encodedwithout reference to another frame and interframes encoded using anotherframe as a reference. An intraframe is also called an I frame inSTAR-based scalable video coding or MPEG video coding or an A frame inUMCTF-based scalable video coding. MPEG video coding uses two types ofinterframes—predictive (P) frame coded using one frame as a referenceand bi-directional (B) frame coded using two frames as a reference. Aninterframe is also called an H frame in scalable video coding.

FIG. 6 shows a first bitstream with the highest resolution (704×576)consisting of an intraframe 610 and three interframes 620. FIG. 6 showsa second bitstream with 352×288 resolution sharing the intraframe 610with the first bitstream. The second bitstream consists of onlyinterframes 640. FIG. 6 shows a second bitstream reorganized with thefirst bitstream and the second bitstream. The reorganized secondbitstream consists of an intraframe 630 obtained from the sharedintraframe 610 in the first bitstream and the interframes 640 in thesecond bitstream.

Sharing the intraframe 610 between the first and second bitstreams meansthat the second bitstream does not contain any intraframe, and theshared intraframe 610 is used in reconstructing a video sequence fromthe second bitstream. That is, a super frame contains the first andsecond bitstreams of FIG. 6, and upon receipt of a request for a videosequence having 352×288 resolution, a video streaming service providercreates the reorganized second bitstream reorganized by truncatingbitstreams other than the intraframe 610 in the first bitstream and thesecond bitstream.

FIG. 7 is a diagram for explaining sharing of an intraframe according toa second embodiment of the present invention. The difference from thefirst embodiment is that a second bitstream of FIG. 7 has a lower framerate than a first bitstream of FIG. 7. The low resolution bitstream hasa lower frame rate than, but the same GOP size as, the high resolutionbitstream. In other words, the number of intraframes in a reorganizedsecond bitstream of FIG. 7 is minimized by increasing the time intervalbetween successive GOPs. Thus, since the number of intraframes in thereorganized second bitstream of FIG. 7 being transmitted afterpredecoding is decreased compared to that in the reorganized secondbitstream of FIG. 6, it is possible to provide a video streaming servicewith a smaller bandwidth. Referring to FIG. 7, the time interval betweenGOPs in the second bitstream is twice as long as that in the firstbitstream since the second bitstream has a half frame rate of the firstbitstream but the same number of frames in each GOP as the firstbitstream. In this case, an H(2) frame in the second bitstream isobtained by intercoding. Like in the first embodiment, the secondbitstream shares an intraframe 710 with the first bitstream for everytwo GOPs of the first bitstream. A super bitstream contains the firstand second bitstreams. Upon receipt of a request for a video sequencehaving 352×288 resolution, a video streaming service provider creates anintraframe 730 by truncating some bits of the shared intraframe 710 andthe reorganized second bitstream consisting of the intraframe 730 andthe interframes 740 in the second bitstream by truncating interframes720 and an unshared intraframe 750 in the first bitstream, and thentransmits the reorganized second bitstream to a video decoder. While itis described above that the number of intraframes in a bitstream beingtransmitted is adjusted using both the time interval between GOPs andsharing of an intraframe, it may be adjusted only by changing the timeinterval between GOPs in bitstreams according to a frame rate withoutsharing any intraframe.

Sharing an intraframe as shown in FIGS. 6 and 7 is an efficient way forremoving redundancies between a plurality of bitstreams having differentresolutions and (or) frame rates. Since the technique according to thefirst or second embodiment of the present invention is usually performedindependently of a video coding algorithm, it can apply to most scalablevideo coding algorithms using both intraframe and interframe codingschemes.

While it is described above that an intraframe in a reorganized secondbitstream is obtained by truncating some bits of an intraframe in afirst bitstream in order to downsample or lower the resolution, it ispossible to obtain the reorganized second bitstream consisting of theintraframe in the first bitstream and interframes in the secondbitstream while maintaining the resolution of the intraframe in thefirst bitstream. This method is useful in non-scalable AVC video codingalgorithms in which it is difficult to adjust the resolution of a frame.When the first and second bitstreams of FIG. 7 are generated using AVCvideo coding, the reorganized second bitstream may be composed of theintraframe 710 in the first bitstream and the interframes 740 in thesecond bitstream. In this case, the video decoder reconstructs anddownsamples the intraframe 710 and uses the interframe as a referenceframe in reconstructing the interframes 740.

As described above, a single video sequence is converted into at leastone video sequence with different resolutions that are then encoded intoscalable bitstreams using scalable video coding, and all or some of thescalable bitstreams (excluding an intraframe) are combined into a superbitstream. A scalable video coding algorithm obtains a low resolutionvideo sequence from a low-pass subband in a wavelet transformed frame.In practice, a low resolution frame obtained by performing wavelettransform tends to be a sharp image, which makes MCTF difficult, thusdegrading video coding efficiency. Thus, conventional MPEG-baseddownsampling may be used to get a softer low-resolution video sequence.In this case, for optimal combination of the MPEG-based downsamplingwith a wavelet-based method, a frame may be downsampled and upsampledusing the wavelet-based method and then downsampled using the MPEG-basedvideo coding.

Meanwhile, in the first and second embodiments of the present inventionshown in FIGS. 6 and 7, a video sequence obtained by performingwavelet-based downsampling on a video sequence used to generate thefirst bitstream is encoded into the second bitstream using scalablevideo coding. In this case, to provide a video streaming service at asecond resolution (352×288), the video streaming service providertruncates unnecessary bits of the super bitstream, leaving the secondbitstream containing no intraframe and the entire interframe or aportion thereof contained in the first bitstream. Then, to obtain asecond resolution intraframe from the intraframe in the first bitstream,the video streaming service provider removes high-pass subbands from afirst resolution (704×576) intraframe consisting of four subbands (onelow-pass subband and three high-pass subbands) with the secondresolution obtained by performing wavelet transform. However, sincedownsampling each frame in the video sequence using the MPEG-basedscheme results in softer images than wavelet-based downsampling,generating the second bitstream from a video sequence obtained byMPEG-based downsampling may provide better coding efficiency andimproved image quality than from a video sequence obtained bywavelet-based downsampling.

In this case, a video sequence may be generated for video coding byusing both wavelet-based downsampling and MPEG-based downsampling,rather than by using only MPEG-based downsampling. That is, a frame maybe downsampled to a lower resolution and upsampled to a higherresolution using a wavelet-based method, and then downsampled to a lowerresolution using an MPEG-based scheme. The details of this method willnow be described with reference to FIG. 8.

FIG. 8 is a diagram for explaining a method for generating a smoothreference frame according to an embodiment of the present invention.

In FIG. 8, D and U respectively denote downsampling and upsampling, andsubscripts W and M respectively denote wavelet- and MPEG-based schemes.F, F_(S), and FL respectively represent high resolution frame, lowresolution frame, and a low-pass subband in the high resolution frame.

In order to obtain a low resolution bitstream, a video sequence is firstdownsampled to a lower resolution and then the downsampled version isupsampled to a higher resolution using a wavelet-based method, followedby MPEG-based downsampling. A low resolution video sequence obtained byperforming the MPEG-based downsampling is then encoded using scalablevideo coding. When a low resolution frame F_(S) is an intracoded frame,the low resolution frame FS is not contained in a bitstream (superbitstream) but obtained from a high resolution intraframe F contained inthe super bitstream. That is, to obtain the smooth low resolutionintraframe F_(S), the high resolution intraframe F is downsampled andthen upsampled using the wavelet-based scheme to obtain approximation ofthe original high resolution interframe F, followed by MPEG-baseddownsampling. The high-resolution intraframe F is subjected to wavelettransform and quantization and then combined into the super bitstream.Some bits of the super bitstream are truncated by a predecoder beforetransmission to a decoder. By truncating high-pass subbands of the highresolution intraframe F, a low-pass subband F_(L) in the high resolutionintraframe F is obtained. In other words, the low-pass subband F_(L) isa downsampled version DW(F) of the high resolution intraframe F. Thedecoder that receives the low-pass subband F_(L) upsamples it using thewavelet-based method and downsamples an upsampled version using theMPEG-based scheme, thus obtaining a smooth intraframe.

The concept is generalized as follows. After obtaining a low resolutionvideo sequence from a high resolution video sequence using adownsampling scheme A, the high resolution and low resolution videosequences are respectively encoded into high resolution and lowresolution bitstreams that are then combined into a super bitstream. Inthis case, the super bitstream consists of all frames in the highresolution bitstream and only interframes (excluding an intraframe) inthe low resolution bitstream. In order to reconstruct a low resolutionvideo sequence from the low resolution super bitstream, unnecessary bitsof the super bitstream are truncated. The unnecessary bits includeinterframes, an unshared intraframe, and high-pass subbands of a sharedintraframe in the high resolution bitstream. After truncation, low-passsubbands of the shared intraframe in the high resolution bitstream andthe interframes in the low resolution bitstream are left in the superbitstream. After reconstructing an intraframe from the low-pass subbandsin the received super bitstream, a video decoder upsamples thereconstructed intraframe and downsamples an upsampled version using thescheme A, thus creating a reference frame to be used in reconstructinglow resolution interframes. Here, the downsampling scheme A may beMPEG-based downsampling filtering described in Doc. N3908, or a bicubicfilter commonly used in image filtering.

This technique applies to coding of videos as well as still images. Forexample, to make a sharp image obtained by wavelet-based downsamplinglook softer, the image may be upsampled using a wavelet-based scheme andthen downsampled using an MPEG-based scheme.

FIG. 9 illustrates a process for providing a video streaming serviceaccording to an embodiment of the present invention.

Referring to FIG. 9, the process includes the following steps:

1. In the illustrative embodiment, a scalable video encoder 910 convertsa single video sequence into a plurality of video sequences with variousresolutions and (or) frame rates and encodes the plurality of videosequences into a plurality of scalable bitstreams.

2. The plurality of scalable bitstreams are combined together into asingle super bitstream. The structure of the super bitstream will bedescribed in more detail later with reference to FIG. 10.

3. The super bitstream is transmitted to a predecoder 920.

4. A user using a video decoder 930 requests a video sequence, andresolution, frame rate and image quality associated with the videosequence. The steps 3 and 4 may be performed in a reverse order.

5. The predecoder 920 that receives the request for resolution, framerate, and image quality truncates unnecessary bits of the superbitstream according to the request.

6. Then, the predecoder transmits the resulting super bitstream to thevideo decoder 930.

7. The video decoder reconstructs a video sequence from the receivedsuper bitstream.

The location of the predecoder 920 may vary depending on the type ofvideo streaming services. In one embodiment, when the predecoder 920 islocated at the video streaming service provider side, the videostreaming service provider includes the scalable video encoder 910 andthe predecoder 920. Upon receipt of a request for a video sequence andresolution, frame rate and image quality associated with the videosequence from the user using the video decoder 930, the predecoder 920predecodes the super bitstream corresponding to the video sequence fortransmission to the video decoder 930. Thus, the super bitstream fromwhich unnecessary bits have been removed are transmitted across anetwork between the streaming service provider and the user.

In another embodiment, the video streaming service provider includes thepredecoder 920 and receives a super bitstream from a content providerincluding the scalable video encoder 910. The super bitstream can betransmitted online on a wired or wireless basis or offline through astorage medium. The received super bitstream is stored in a storagedevice (not shown) at the video streaming service provider. Upon receiptof a request for a video sequence from the user using the video decoder930, the video streaming service provider searches the storage devicefor a super bitstream that matches the requested video sequence, andtruncates unnecessary bits of the found super bitstream for transmissionto the user.

In yet another embodiment, the predecoder 920 is located at the userside. When the video streaming service provider also includes thepredecoder 920, the predecoder 920 operates in the same manner as whenit is located at the video streaming service provider side. When thevideo streaming service provider does not include the predecoder 920,the scalable video encoder 910 at the video streaming service providerside generates a super bitstream from a video sequence. Upon receiving auser's request for a video sequence, the video streaming serviceprovider transmits a super bitstream corresponding to the video sequenceto the user on a wired or wireless basis.

In a further embodiment, the predecoder 920 truncates unnecessary bitsof the super bitstream according to the desired resolution, frame rate,and image quality and reconstructs the video sequence from the resultingsuper bitstream. When the video sequence with the desired resolution andframe rate does not exist in the super bitstream, bits of the remainingbitstreams excluding one of higher resolution and frame rate bitstreamsare truncated. The selected bitstream may have resolution closest to thedesired resolution.

FIG. 10 shows the structure of a super bitstream according to anembodiment of the present invention.

To obtain a super bitstream, one video sequence is first converted intoa plurality of video sequences with various resolutions that are thencompressed into a plurality of bitstreams using a scalable video codingscheme. The plurality of bitstreams are combined together into a superbitstream. Each bitstream contains motion vectors obtained during videocompression and information on wavelet coefficients which have beensubjected to embedded quantization.

The super bitstream is composed of a super bitstream header 1000 andfirst through n-th bitstreams 1010-1 through 1010-n. The super bitstreamheader 1000 contains various necessary information on the superbitstream. For example, the information may include the number ofbitstreams in the super bitstream and information about sharing of anintraframe.

The first bitstream 1010-1 contains information obtained by encoding theoriginal highest resolution video sequence. A bitstream header 1020contains various necessary information on the first bitstream includinglength, resolution, frame rate, and the number of GOPs. A GOP header1030 contains GOP information such as the number of frames in each GOPor a GOP size. The GOP header 1030 is followed by frame #1 through frame#n (1040-1 through 1040-n). When one GOP ends, the next GOP also beginswith a GOP header followed by frames.

The frame #1 1040-1 is the first frame of a GOP that is an intracodedframe (called an I frame or an A frame) in most scalable video codingalgorithms. A frame header 1050 contains information specifying the typeof the frame and necessary information about the frame such as length,followed by a low-pass subband 1060 and a high-pass subband 1070. Thelow-pass subband 1060 represents a texture image obtained by performingembedded quantization on a low-frequency component (LL image) obtainedby applying wavelet transform to the frame. The high-pass subband 1070represents a texture image obtained by performing embedded quantizationon high frequency components (LH and HH images) obtained using thewavelet transform. The frame #1 1040-1 further includes motion vectorinformation (not shown).

In connection with the structure of the super bitstream, sharing of alow frequency intraframe as shown in FIGS. 6 and 7 will now bedescribed. The first bitstream 1010-1 with the highest resolutioncontains information about all frames in the original video sequence,while the second bitstream 1010-2 with lower resolution contains nointraframes. The super bitstream can be preceded into the secondbitstream 1010-2 in various ways. In one embodiment, the remainingportions of the super bitstream excluding intraframes of the firstbitstream 1010-1 and the second bitstream 1010-2 are truncated. Afterreconstructing and downsampling intraframes of the first bitstream, theframes of the second bitstream are reconstructed at a decoder side.

In another embodiment, remaining portions of the super bitstreamexcluding the intraframes of the first bitstream 1010-1 and the secondbitstream 1010-2 are truncated, and high-pass subbands from each of theintraframes in the first bitstream 1010-1 are truncated. In a decoderpart, the remaining low-pass subbands of intraframe in the firstbitstream 1010-1 are reconstructed, and the reconstructed low-passsubbands are then upsampled and downsampled in order to create areference frame used in reconstructing other interframes. The lattermethod reduces the amount of data in a bitstream being transmitted tothe decoder side compared to the former.

In concluding the detailed description, those skilled in the art willappreciate that many variations and modifications can be made to theexemplary embodiments without substantially departing from theprinciples of the present invention. While the present invention hasbeen particularly shown and described with reference to a videostreaming service provider using a plurality of scalable video codingalgorithms, video streaming services may be delivered using acombination of scalable and non-scalable video coding schemes.Therefore, the disclosed exemplary embodiments of the invention are usedin a generic and descriptive sense only and not for purposes oflimitation. It is to be understood that various alterations,modifications and substitutions can be made therein without departing inany way from the spirit and scope of the present invention, as definedin the claims which follow.

As described above, the present invention enables video coding forsimulcasting with good image quality and high video coding efficiencyand video streaming services using the video coding.

In addition, according to the present invention, the efficiency of avideo coding algorithm for the video streaming services can be improved.

Further, the present invention improves the quality of a video stream aswell as individual (still) images such as provided by video streamingservices.

1. A video encoding method comprising: encoding first and second videosequences into first and second bitstreams using scalable video coding,wherein at least one of resolution, frame rate, and image quality of thesecond video sequence is different from that of the first videosequence; and combining the first and second bitstreams into a superbitstream.
 2. The method of claim 1, further comprising encoding thirdthrough n-th video sequences into third through n-th bitstreams usingscalable video coding, wherein the super bitstream further includes thethird through n-th bitstreams and at least one of the resolution, framerate, and image quality of each of the third through the n-th videosequences is different from those of the first and second videosequences, where n is a natural number greater than
 1. 3. The method ofclaim 2, wherein each of the second through the n-th video sequences isobtained by downsampling each frame in the first video sequence into alower resolution or by removing some of the frames in the first videosequence in order to reduce a frame rate.
 4. The method of claim 3,wherein the downsampling of each frame in the first video sequence isperformed using MPEG-based downsampling.
 5. The method of claim 3,wherein the downsampling is performed by downsampling each frame in thefirst video sequence using a wavelet-based method, upsampling adownsampled version using the wavelet-based method, and downsampling anupsampled version using an MPEG-based scheme.
 6. The method of claim 1,wherein the second bitstream in the super bitstream does not contain anyintracoded frame.
 7. The method of claim 1, wherein the second videosequence has a lower frame rate but longer time interval between groupsof pictures (GOPs) than the first video sequence.
 8. A video encodingmethod comprising: encoding first through n-th video sequences intofirst through n-th bitstreams wherein n is a natural number greater than1 and at least one of resolution, frame rate, and image quality of eachof the second through the n-th video sequences is different from that ofthe first video sequence; and combining the first through the n-thbitstreams together into a super bitstream.
 9. The method of claim 8,wherein at least one of the first through the n-th bitstreams isgenerated using scalable video coding.
 10. The method of claim 8,wherein at least one of the first through the n-th bitstreams isgenerated using scalable video coding and at least one of the firstthrough n-th bitstreams is generated using Advanced Video Coding (AVC).11. The method of claim 8, wherein a bitstream having the highestresolution among the first through the n-th bitstreams is generatedusing AVC video coding.
 12. The method of claim 8, wherein a bitstreamhaving the lowest resolution among the first through the n-th bitstreamsis generated using AVC video coding.
 13. A super bitstream formatcomprising: a first bitstream generated by encoding a first videosequence having a highest resolution; and second through n-th bitstreamsgenerated by respectively encoding second through n-th video sequenceshaving resolution, frame rate, and image quality, at least one of whichis different from that of the first video sequence where n is a naturalnumber greater than
 1. 14. The super bitstream format of claim 13,wherein the first bitstream is generated using scalable video coding.15. The super bitstream format of claim 13, wherein at least one of thesecond through the n-th bitstreams does not contain any intraframe. 16.A video encoder comprising: a video encoding unit encoding first throughn-th video sequences wherein at least one of resolution, frame rate, andimage quality of each of the second through the n-th video sequences isdifferent from that of the first video sequence; and a super bitstreamgenerator generating a super bitstream including first through n-thbitstreams generated by the video encoding unit and necessary headerinformation; wherein n is a natural number greater than
 1. 17. The videoencoder of claim 16, wherein the video encoding unit encodes the firstthrough the n-th video sequences using scalable video coding.
 18. Thevideo encoder of claim 16, wherein the video encoding unit encodes thefirst through the n-th video sequences using Advanced Video Coding(AVC).
 19. The video encoder of claim 16, wherein the video encodingunit encodes the at least one of the first through the n-th videosequences using scalable video coding and the at least one of the firstthrough n-th bitstreams using AVC video coding.
 20. The video encoder ofclaim 16, further comprising a converter converting the first videosequence into the second through the n-th video sequences bydownsampling frames in the first video sequence in order to lower theresolution and removing some of the frames in order to reduce a framerate.
 21. The video encoder of claim 16, wherein the video encoding unitencodes the first through the n-th video sequences into the firstthrough n-th bitstreams, at least one of which does not contain anyintraframe.
 22. A predecoding method comprising: receiving a request fora video sequence with predetermined resolution, frame rate, and imagequality; truncating a portion of a super bitstream including a pluralityof bitstreams with different resolutions and frame rates associated withthe requested video sequence, so that the super bitstream has the sameresolution, frame rate, and image quality as requested; and transmittingthe resulting super bitstream to a decoder.
 23. The method of claim 22,wherein the truncating of the portion of the super bitstream comprisesselecting a bitstream that matches the requested resolution and framerate and truncating a remaining bitstream excluding the selectedbitstream and unnecessary bits of the selected bitstream so that thesuper bitstream has the same resolution, frame rate, and image qualityas requested.
 24. The method of claim 22, wherein the truncating of theportion of the super bitstream comprises selecting a bitstream thatmatches the requested resolution and frame rate and truncating aremaining bitstream excluding the selected bitstream and a bitstreamsharing intraframes with the selected bitstream and unnecessary bits ofthe selected bitstream and the bitstream sharing the intraframes so thatthe super bitstream has the same resolution, frame rate, and imagequality as requested.
 25. A predecoder comprising: a request receiverreceiving a request for a video sequence with predetermined resolution,frame rate, and image quality; a super bitstream truncating portiontruncating a portion of a super bitstream including a plurality ofbitstreams with different resolutions and frame rates associated withthe requested video sequence, so that the super bitstream has the sameresolution, frame rate, and image quality as requested; and atransmitter sending a resulting super bitstream to a decoder.
 26. Amethod for reconstructing a video sequence, comprising: receiving acompressed video sequence, performing inverse quantization and inversetransform on the compressed video sequence, and reconstructing anintraframe; filtering the intraframe so that the intraframe is used as areference frame in reconstructing interframes; and reconstructing theinterframes by using a filtered version of the intraframe as areference.
 27. The method of claim 26, wherein the filtering of theintraframe is performed by downsampling or upsampling the intraframe.28. The method of claim 26, wherein the filtering of the intraframe isperformed by upsampling the reconstructed intraframe using awavelet-based method and downsampling an upsampled version of theintraframe using an MPEG-based scheme.
 29. An apparatus forreconstructing an original video sequence from a compressed videosequence, the apparatus comprising: a receiver receiving the compressedvideo sequence; a decoder reconstructing an intraframe by decoding thecompressed video sequence; and a filter filtering the reconstructedintraframe so that the intraframe is used as a reference inreconstructing interframes, wherein the decoder reconstructs interframesby decoding the compressed video sequence using a filtered version ofthe intraframe as a reference.
 30. The apparatus of claim 29, whereinthe filter performs filtering by downsampling or upsampling thereconstructed intraframe.
 31. The apparatus of claim 30, wherein thereconstructed intraframe is upsampled using a wavelet-based method andan upsampled version of the intraframe is downsampled using anMPEG-based scheme.
 32. The apparatus of claim 31, wherein the decoderreconstructs an intraframe by performing inverse quantization on thecompressed video sequence and inverse spatial transform on transformcoefficients obtained by performing the inverse quantization, the filterfiltering the reconstructed intraframe, and the decoder reconstructsinterframes using a filtered version of the intraframe as a referenceafter performing the inverse quantization on the compressed videosequence and the inverse spatial transform on the transform coefficientsobtained by performing the inverse quantization.
 33. A recording mediumhaving a computer readable program recorded therein, the program forexecuting the method of claim
 1. 34. A method for filtering an imageobtained by wavelet-based downsampling comprising: performingwavelet-based upsampling on the image obtained by the wavelet-baseddownsampling; and downsampling an upsampled version of the image using apredetermined scheme.
 35. The method of claim 34, wherein thedownsampling is performed based on Advanced Video Coding (AVC).